Tracking objects in three dimensions (3D) involves calculating the global position and orientation of the object from captured sensor data, and where the object is articulated (such as the human body, a human hand or a laptop computer), calculating positions of one or more joints. There is an increasing need to improve accuracy of such tracking to give better control of systems which use the tracking data, such as augmented reality systems, virtual reality systems, medical equipment, natural user interfaces, game systems and others. Where tracking fails or is inaccurate this produces problems in the systems that use the tracking data.